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SPECIFICATION 

REMOTE IMAGE DISPLAY METHOD, IMAGE CAPTURING DEVICE 
AND METHOD, AND PROGRAM THEREFOR 

5 

TECHNICAL FIELD 

The present invention relates to a remote image display method for 
image sensing of a panoramic object by an image sensor, that is, by a camera 
(hereinafter referred to simply as camera) that outputs video signals, and for 
10 displaying the captured images to a user at a place remote from the camera; 
the invention also pertains to an image capturing device and method therefor, 
and an image capturing program. 

BACKGROUND ART 

15 For the purpose of observing the traffic on the road, it is now a general 

practice to set a panhead camera to monitor the traffic conditions; video 
signals obtained by shooting with the camera are sent over a communication 
line to a monitor set placed at a remote location, where the received video 
signal are played back to display the traffic conditions on a display. The 

20 watcher manipulates, while watching the display, an operating button or lever 
of a remote controller to send a control signal for changing the shooting 
direction (horizontal and vertical directions) of the monitor camera, or/and a 
control signal for changing the zoom amount of a zoom lens of the camera 
over the communication line to a control mechanism of the monitor camera to 

25 control it for changing its shooting direction and shooting condition (refer to, 
for example, Japanese Patent Application Koukai publication No. 205775/99 
(published on July 30, 1999). 
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There is offered a service what is called video live through a panhead 
camera set on the roof of a high-rise building; a user accesses the camera from 
his personal computer via the Internet or similar communication network to 
occupy the camera for a predetermined time, receives shooting video signals 
5 of the camera by the personal computer to play back the video signals to 
provide a display their images on the display, and while watching the image 
on the display, the user operates the personal computer to remotely control the 
camera shooting direction or the zoom amount of the lens to enjoy seeing 
respective parts of a wide^ange panoramic object. 

10 It is possible for a parent to control a personal computer at his home or 

office to access a camera set up in kindergarten to play back video signal from 
the camera for display on the personal computer and to remotely control the 
shooting direction or zoom amount of the camera to observe his child's 
movements under various circumstances. 

15 Further, when video signals from a security camera set up in a 

convenience store are played back to display the images on a display at a 
remote place, it is possible for a watchman to remotely control the shooting 
direction or zoom amount of the security camera while watching the images 
being displayed. 

20 Such remote control of a camera to display video signals therefrom 

can be used for various purposes. In the prior art, the shooting direction of 
the camera itself is changed by mechanical control, or the control for 
mechanically moving the zoom lens is effected by a button (key) operation or 
level operation using a personal computer. Moreover, while one user 

25 remotely controls the camera, other users cannot controls the camera; that is, 
the camera is occupied by one user for a predetermined time, for instance. 
Since the control for obtaining a desired image by a conventional 
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remote image display scheme is remote control of the camera itself by 
operating a button (key) or lever, the operability of the camera is poorer than 
in the case of manually operating the camera in one's hand or on the panhead 
to obtain a desired image, that is, the image thus obtained becomes a little 
5 different from the desired one in some cases. 

An object of the present invention is to provide a remote image 
display method that permits displaying a desired remote image in the same 
way a user directly operates the camera by his hand, and an image capturing 
device and method therefor, and an image capturing program. 

10 

DISCLOSURE OF THE INVENTION 

According to the remote image display method of the present 
invention: a remote image sensing device performs image sensing of a 
panoramic object, and sends a video signal of a part of image-sensed 

15 panoramic object via an image capturing device to image display means at a 
place different from the remote image sensing device; the image display 
means lays back and display the received video signal as an image of a part of 
said panoramic object; a camera-equipped portable terminal at a position 
where the display on said image display means can be seen performs image 

20 sensing of its surroundings, and the surrounding video signal to said image 
capturing device; the image capturing device obtains variation information 
about the surrounding image from the camera-equipped portable terminal 
based on a previous received surrounding video signal and the current 
received surrounding video signal, then obtains from the remote image 

25 sensing device a video signal of a part of the panoramic object changed 

corresponding to a change of the surrounding image based on the variation 
information, and sends the obtained image signal to the image display means. 
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The panoramic object is sufficiently wide to such an extent as not to 
be suitable for displaying as an image of one display frame of the image 
display means, or it can be displayed as the image of one display frame to 
some extent but it is equal to the range of a visual angle over which each part 
5 of the display needs to be seen in detail, and the field angle is 360 degree or 
less, and the images to be displayed need not always be continuous. There 
are two schemes for the image capturing device to obtain the video signal of a 
changed part of the panoramic object from the remote image sensing device. 
According to the one scheme: variation information on the image of the 

10 current surrounding video signal (hereinafter referred to as the current image) 
with respect to the mage of the previous surrounding video signal (hereinafter 
referred to as the previous image) received from the camera-equipped 
portable terminal is detected by variation detecting means from the received 
surrounding video signal; a capturing signal for obtaining a video signal of a 

15 part of the panoramic object is generated by capturing signal generating 

means based on the detected variation information; the generated capturing 
signal is sent to the remote image sensing device; and the video signal 
received by the remote image sensing device is sent by image relay means to 
the image display means. With this scheme, the image capturing device 

20 sends the capturing signal to the remote image sensing device and receives the 
video signal of the changed part of the panoramic object from the remote 
image sensing device to thereby obtain an image of the video signal of the 
changed part of the panoramic object. 

According to the other scheme, the user operates the camera-equipped 

25 portable terminal in his hand to shoot a part of the panoramic object while 
watching the image of a part of the panoramic object displayed on the image 
display means, by which it is possible to remotely shoot and display a desired 
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part of the panoramic object; in this case, the camera can be operated in the 
same way the user himself operates the camera at the place where he can see 
the panoramic object directly. Hence, operability of the camera is excellent 
and no deviation is hard to occur between the desired part of the panoramic 
5 object and the displayed image. 

BRIEF DESCRIPTION OF THE DRAWINGS 

Fig. 1 is a diagram illustrating an example of the system configuration 
for explaining the principles of the present invention. 
10 Fig. 2 is a diagram for explaining an image sensor device of an 

omnidirectional video camera and the extraction of an image signal of an each 
partial object. 

Fig. 3 A is a diagram showing, by way of example, how captured 
images move on the image sensor when a camera-equipped portable terminal 
15 is turned in the direction of taking pictures. 

Fig. 3B is a diagram showing images before and after they move in 
Fig. 3A. 

Fig. 4 is a block diagram illustrating an example of the system 
configuration of Embodiment 1 . 
20 Fig. 5 is a diagram depicting an example of the functional 

configuration of each of variation detecting means 12 and capturing signal 
generating means 13 in Fig. 4. 

Fig. 6 is a diagram showing an example of the operation procedure of 
the entire system shown in Fig. 4. 
25 Fig. 7 is a flowchart showing an example of the procedure of an image 

capturing device 1 in Fig. 4. 

Fig. 8 is a block diagram illustrating an example of the system 
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configuration of Embodiment 2. 

Fig. 9 is a flowchart showing an example of the procedure of the 
image capturing device 1 in Fig. 8. 

Fig. 10 is a diagram, by way of example, the relationships between a 
5 remote image sensing device 2 using plural camera devices and a panoramic 
object 6. 

Fig. 11 is a block diagram illustrating an example of the system 
configuration of Embodiment 3. 

Fig. 12 is a diagram for explaining an image sensor device of the 
10 camera device in Fig. 11 and the extraction of a captured signal of an partial 
object. 

Fig. 13 is a block diagram showing an modified form of the remote 
image capturing device 1 in Fig. 11. 

Fig. 14 is a diagram showing an example of a concrete functional 
15 configuration of capturing signal generating means 13 and an example of 
stored contents of camera direction storage means 19 in Fig. 11. 

Fig. 15 is a flowchart showing an example of the procedure of a 
camera determining part 13b in Fig. 14. 

Fig. 16 is a diagram showing another example of the stored contents 
20 of the camera direction storage means 19. 

Fig. 17 is a block diagram illustrating an example of the system 
configuration of Embodiment 4. 

Fig. 18 is a block diagram illustrating an example of the system 
configuration of Embodiment 5 . 
25 Fig. 19 is a diagram showing an example of the outward appearance of 

a panhead camera in Fig. 18. 

Fig. 20 is a block diagram illustrating an example of the system 
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configuration of Embodiment 6. 

Fig. 21 is a block diagram illustrating an example of he functional 
configuration of the image capturing device 1 in Fig. 20. 



5 BEST MODE FOR CARRYING OUT THE INVENTION 
Description of Principles of the Invention 

The principles of the present invention will be described with 
reference to Fig. 1. 

This example uses an omnidirectional camera as a remote image 

10 sensing device 2. The omnidirectional camera is a digital video camera, 

which is an omnidirectional image sensor capable of image sensing of objects 
360-degree surroundings; refer to, for example, Kazumasa Yamazaki, 
"Principle and Characteristics of Omnidirectional camera with a mirror," 
Computer Vision and Image Media, 125-21, P. 155~ 160, January 19, 2001. 

15 The omnidirectional camera is commercially available; by which images of a 
360-<Iegree panoramic object, with a predetermined reference direction at zero 
degree as shown in Fig. 2, for instance, are formed on a rectangular 
two-dimensional image sensor device 7, such as CCD, from which 
photoelectric conversion signals corresponding to respective pixels can be 

20 extracted. 

In Fig. 1 the remote image sensing device 2 is capable of image 
sensing of a 360-degree surrounding panoramic object 6. In this example a 
captured video signal of a part of the panoramic object by the remote image 
sensing device 2, for example, a part (hereinafter referred to as a partial 
25 object) 61 in a direction 180 degrees away from the reference, that is, each 
pixel signal of an area 71 in the image sensor device 7 in Fig. 2 is extracted 
and sent over a transmission line 4 to the image capturing device 1 . 
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The image capturing device 1 sends the received captured video signal 
of the partial object over a transmission line 5 to a camera-equipped portable 
terminal 3 that is located at a place different from the remote image sensing 
device 2 and serves also as image display means. This example uses a 
5 camera-equipped portable telephone as the camera-equipped portable terminal 
3, but it may be a camera-equipped PDA (Personal Digital Assistance) or 
camera-fixed PDA terminal as long as it is equipped with a digital video 
camera and communicating means and can be manually operated to take 
pictures. To distinguish between the transmission lines 4 and 5, they will 

10 hereinafter be referred to as a remote transmission line 4 and a 

portable-associated transmission line 5, respectively. Both of the 
transmission lines 4 and 5 may be dedicated transmission lines or 
transmission lines that pass through one or more communication networks, 
such as the Internet, or wireless/wire public telecommunication network. 

15 Incidentally, in this example, the camera-equipped portable terminal 3 

is also provided with image display means, and the captured video signal of 
the partial object 61, sent from the remote image sensing device 2 and 
received from the image capturing device 1, is played back by the image 
display means and displayed on its display surface 32a. 

20 A user holds the camera-equipped portable terminal 3 in his hand and 

takes pictures of a surrounding object 8 by a camera (portable image sensing 
means) 33 of the terminal 3; as he sees the partial object 61 in the panoramic 
object 6 displayed on the display surface 32a, if he wants to see another 
partial object in the panoramic object 61 that he wishes to display at a remote 

25 place, then the user operates the camera-equipped portable terminal 3 to direct 
its camera 33 to take pictures of the partial object desired to see. For 
example, in the case of taking pictures of a partial object 62 in a direction 45 
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degree away to the right from the partial object 62, the direction 33a of the 
camera 33 of the camera -equipped portable terminal 3 is turned 45 degrees to 
the right as indicated by the one-dot chain line in Fig. 1. 

The video signal of the surrounding object being image sensed by the 
5 camera 33 of the camera -equipped portable terminal 3 (which video signal 
will hereinafter be referred to as a surrounding video signal) is sent over the 
portable-associated transmission line 5 to the image capturing device 1 . The 
image capturing device 1 detects a change in the surrounding image being 
shot by the camera-equipped portable terminal 3 from the previous and 

10 current surrounding video signals. 

The range over which the camera-equipped portable terminal 3 is 
shooting the surrounding object 8 is a shooting range 8b shifted to right from 
that 8a as shown in Fig. 3 A. Accordingly, the received surrounding video 
signal at the time of seeing the reproduced image of the partial object 61 is the 

15 video signal of the shooting range 8a shown at the upper side of Fig. 3B, but 
the received surrounding video signal with the shooting direction of the 
camera-equipped portable terminal 3 turned 45 degrees to right is the video 
signal of the shooting range 8b shown at the lower side of Fig. 3B. 
Comparison of the both received surrounding video signals indicates that the 

20 same image part shits from right to left in the drawing, from which it is 

possible to detect that the shooting direction of the camera-equipped portable 
terminal 3 has been turned to right. 

The image capturing device 1 detects variation information about the 
received surrounding video signal with respect to the previous one, then based 

25 on the variation information, changes the video signal to be extracted from the 
video signal being shot by the remote image sensing device, and instructs the 
remote image sensing device 2 so that the partial object in the panoramic 



10 

object 6 corresponding to the extracted video signal changes corresponding to 
the change in the image of the received surrounding video signal. In the 
case where the shooting direction of the camera-equipped portable terminal 3 
is turned 45 degrees to right in the above example, the image capturing device 
5 2 instructs the remote image sensing device 2 via the remote transmission line 
4 to extract the video signal corresponding to the partial object 62 to which 
the partial object 61 of the panoramic object 6 has turned 45 degrees to right 
and to extract the video signal of the area 72 to which the area 71 on the 
image sensor device 7 in Fig. 2 has turned 45 degrees to right. 

10 Upon receiving the instruction, the remote image sensing device 2 

extracts the video signal from the area 72 of the image sensor 7, and sends the 
extracted video signal to the image capturing device 1 via the remote 
transmission line 4, and the image capturing device 1 sends the received video 
signal to the camera-equipped portable terminal 3 via the portable-associated 

15 transmission line 5. The camera-equipped portable terminal 3 displays on its 
display surface 32a the reproduced image corresponding to the partial object 
62. 

The remote image sensing device 2 always shoots the panoramic 
object 6 and responds to the capturing instruction from the image capturing 
20 device 1 to send at all times the video signal of the partial object via the image 
capturing signal to the camera-equipped portable terminal 3 at the rate of 30 
frames per second, for instance. The camera-equipped portable terminal 3 
also sends its surrounding video signal to the image capturing device 1 at all 
times. 

25 As will be seen from the above, according to the method of the present 

invention, remote shooting and displaying of a desired partial object can be 
achieved by operating the camera-equipped portable terminal 3 to shoot the 
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part of the panoramic object 6 desired to display while directly watching the 
panoramic object — this provides enhanced operability of the camera and 
makes it unlikely to incur the deviation between the desired partial object and 
the reproduced image of its video signal. For example, in the case of 
5 remotely controlling the camera by operating keys of a portable terminal such 
as a portable telephone, the operability is poor because of the key operation, 
and since the keys are small and closely packed, an erroneous operation is 
likely to occur, but the method of the present invention is free from such a 
possibility. 

10 EMBODIMENT 1 

Fig. 4 illustrates in block form an example of the system configuration 
of Embodiment 1 of the present invention. In this application the same 
reference numerals denote parts of the same function and no duplicate 
description will be given of them. 

15 The image capturing device 1 is provided with: surrounding image 

receiving means 1 1 that receives a surrounding video signal sent from the 
camera-equipped portable terminal 3 over the portable-associated 
transmission line 5; variation detecting means 12 that detects a variation in the 
image of the received surrounding video signal as variation information; 

20 capturing signal generating means 13 that generates a capturing signal based 
on the detected variation information; sending means 14 which sends the 
generated capturing signal to the remote image sensing device 2 over the 
remote transmission line 4; and image relay means 15 that receives the 
captured video signal sent from the remote image sensing device 2 over the 

25 remote transmission line 4 and passes it to the camera-equipped portable 
terminal 3 over the portable-associated transmission line 5. 

The remote image sensing device 2 is provided with: an 
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omnidirectional camera 21 as remote image sensing device means for image 
sensing of a panoramic object; signal receiving means 23 that receives the 
capturing signal sent from the image capturing device 1 over the remote 
transmission line 4; image extracting means 24 that extracts the captured 
5 video signal of a partial object from the omnidirectional camera 21; and 
image sending means 22 that sends the captured video signal to the image 
capturing device 1 over the remote transmission line 4. 

The camera-equipped portable terminal 3 is provided with: image 
receiving means 31 that receives the captured video signal sent from the 

10 remote image sensing device 2 over the portable-associated transmission line 
5; image display means 32 that plays back and displays the received captured 
video signal; a digital video camera 33 that is portable image sensing means 
for image sensing of surrounding objects; and sending means 34 that sends 
the surrounding video signal fed thereto from the camera 33 to the image 

15 capturing device 1 over the portable-associated transmission line 5. 

The image extracting means 24 of the remote image sensing device 2 
extracts from the omnidirectional camera 21a captured video signal of one 
frame that is determined by the received capturing signal. For example, 
based on the capturing signal, the image capturing means extracts a sequence 

20 of signals corresponding to pixels (x + a p i, y + a p i) (where i = 0, ±1, . . ., ±1, J = 
0, ±1, . . ., ±J, 21 being the number of x-direction pixels corresponding to one 
frame and 2J the number A, of y-direction pixels corresponding to one frame) 
about a pixel position (xi, y{) in the image sensor device shown in Fig. 2, and 
the thus extracted signal is output as a captured video signal of the area 7 1 . 

25 That is, in this example the center position of the partial object desired to 
extract and the zoom amount are specified by the capturing signal. In this 
example the capturing signal is composed of reference pixel position signals x, 
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y and a zoom in/out signal a p . 

The captured video signal is sent via the image capturing device to the 
camera-equipped portable terminal 3, and the image of the partial object in 
the panoramic object being image sensed by the remote image sensing device 
5 2, in this example, the image of the partial object 62 in Fig. 1, is displayed on 
the image display means 32 of the camera-equipped portable terminal 3. 

The variation detecting means 12 of the image capturing device 1 
detects, from the received surrounding video signal, information about a 
variation between the image of the current video signal (which image will 

10 hereinafter be referred to as the current image) and the image of the previous 
video signal received from the camera-equipped portable terminal 3. For 
example, as depicted in Fig. 5, the received surrounding video signal is stored 
in a receiving buffer memory 12a, before which the surrounding video signal 
previously received and stored in the receiving buffer memory 12a is stored in 

15 a previous frame memory 12b. The surrounding video signals stored in the 
memories 12a and 12b are both input to a variation detecting part 12c. The 
variation detecting part 12c detects the direction of movement from the 
previous image to the current one or the degree of movement (distance) 
or/and a change in the size of the object of the current image with respect to 

20 the previous image, i.e., the zoom amount, are detected. The variation 
information can be detected, for example, by techniques for analyzing the 
camera operation during shooting, that is, what is called camera work. The 
camera work analysis technique is disclosed, for example, in Yukinobu 
TANIGUCHI, et at., "PanoramExcerpts: Video Cataloging by Automatic 

25 Synthesis and Layout of Panoramic Images," IECEJ Journal D-II, Vol. 
J82-D-II No. 3, PP. 392, March 1999 and a book supervised by Kenji 
KOGURE, written by Kazuhiko YAMAMORI, "Future Network Technology 
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Series, Media Processing Techniques 4," 20, Denki-Tsuushin Kyoukai, First 
Edition, November 10, 1999. 

A description will be given of an example of the scheme for analyzing 
camera works of side-to-side operation (pan), up-and-down operation (tilt) 
5 and operation of changing the angle of view by the camera lens (zoom). Let 
the previous image f (x, y) and the current image by f *(x\ y') 5 where 

(x\ y') = (ax+d x , ay+d y ) 
that is, let a, d x and d y represent parameters for explaining zoom, pan and tilt, 
respectively. The parameters a, d x and d y are calculated which minimize the 

10 following square error between the images f(x, y) and f '(x\ y'): 
(1/N) Z x , y {f(x, y)-f(x',y')} 2 . 
These parameters a, d x and d y are used as the variation information. In other 
words, the variation detecting part 12c calculates, for example, a, d x and d y to 
obtain the variation information; that is, a is zoom in/out information, and d x 

15 and d y direction information. In the case of detecting only the side-to-side 
turning of the camera-equipped portable terminal 3, the parameters need only 
to be set at a=l and d y =0; in the case of detecting only the up-and-down 
turning action, a=l and d x =0; in the case of detecting only a change in the 
zoom, d x =d y =0; and in the case of detecting only the turning action, a= 1 . 

20 The capturing signal generating means 13 generates, from the 

variation information detected by the variation detecting means 12 and a 
previous capturing signal, a capturing signal that is used to obtain the 
captured video signal of the partial object of the panoramic objection. For 
example, as shown in Fig. 5, in the capturing signal generating means 13, 

25 capturing signals x, y and a p , for instance, generated previously and currently 
used are stored in a previous signal memory 13a; these signals x, y and a p and 
the variation information, for example, d x , d y and a x , are calculated by x + d x 
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— > x in an adding part 13b, y +d y — > y in an adding part 13c, and a p xa^a p 
in a multiplying part 13d, and they are complemented to generate new 
capturing signals x, y and a p , which are provided to the previous signal 
memory 13a to update the capturing signals stored therein, and at the same 
5 time, they are output to the sending means 14. While the image capturing 
device 1 does not perform the image capturing processing, there stored in the 
previous signal memory 13a, as initial values, predetermined capturing 
signals, for example, the center, x, y, of the sensor device in Fig. 2 and a half 
value, a p , of the maximum zoom amount; upon the image capturing device 1 

10 starting the capturing processing, the initial value signals x, y and a p in the 
previous signal memory 13a are sent as capturing signals to the remote image 
sensing device 2. 

When the value of the variation information is small, for example, 
when d x = 1, the image of the partial object of the panoramic object displayed 

15 on the camera-equipped portable terminal 3 remains substantially unchanged; 
to cope with this, it is advisable to provide round-off pats 13e, 13f and 13g, by 
which if d x , d y and a are smaller than their predetermined values, d x and d y are 
rounded off to 0s and a to 1, which are provide to the adding parts 13b and 
13c and the multiplying part 13d. The above-mentioned predetermined 

20 values set in the discarding parts 13e, 13f and 13g differ among the discarding 
parts and according to the mode of use of this remote display system. In the 
case of displaying a partial object in a distant scene, the predetermined values 
are chosen relatively large, and in the case of using the system to prevent 
crime in a relatively space, they are chosen relatively small; as for d x , for 

25 instance, it is considered to be rounded down to 0 when it is 5 or smaller at 
minimum. 

In general, updating of the capturing signals to be sent to the remote 
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image sensing device 1 1 upon each reception of one frame is not preferable 
since it increases the amount of processing, and even if updating is carried out 
for each frame, the user hardly perceives a change in the received 
partial-object image. Accordingly, the capturing signals may be sent to the 
5 remote image sensing device 11 at appropriate time intervals. This time 

interval may be determined properly based on the usage pattern of the system: 
for example, every 1/10 sec when conditions of the object change relatively 
rapidly as in the case of traffic monitoring; when the object is scenery, every 
1/3 sec, for instance, and in some cases, every several seconds. To this end, 

10 an update instructing part 12d is provided, for example, in the variation 
detecting means 12, and the receiving buffer memory 12a responds to an 
instruction from the update instructing part 12d to capture the surrounding 
video signal of one frame at preset update time intervals, and the variation 
detecting part 12c detects variation information at the update time intervals. 

15 Alternatively, the contents of the receiving buffer memory 12a are updated for 
each frame of surrounding video signals and, upon each detection of variation 
information, the contents of the receiving buffer memory 12a are transferred 
to the previous frame memory 12b. When all pieces of variation information 
detected in the variation information detecting part 12c are zero, that is, when 

20 d x = d y = 0 and a = 1 in this example, it is not necessary to generate and send 
the capturing signals to the remote image sensing device, in which the 
capturing signals need not be generated, either. The capturing signals may 
selectively be generated and transmitted in correspondence only to those 
pieces of the variation information which have undergone changes. 

25 In the remote image sensing device 2, the capturing signals received 

by the signal receiving means 23 are input to the image extracting means 24 
as shown in Fig. 4, and the previously received capturing signals stored in a 
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storage part 24a with the newly received capturing signals. Based on the 
thus updated capturing signals in the storage part 24a 5 the image extracting 
means 24 extracts the video signal of the partial object in the panoramic 
object being shot by the omnidirectional camera 21. For example, the output 
5 of the image sensor device 7 of the omnidirectional camera 21 changes, as 
shown in Fig. 2, from the state where video signals of a one-frame area 71 
with the pixel (x, y) at its center (reference) are always extracted based on the 
yet-to-be capturing signals (reference pixel position signals) to the state where 
video signals of a one-frame area 72 with the pixel (x\ y') at its center are 

10 always extracted based on the updated capturing signals. In the example of 
Fig. 2, the shooting direction of the camera-equipped portable terminal 3 does 
not change vertically nor does the zoom amount and the shooting direction is 
turned only to the right, that is, the variation information d y = o and a = 1 and 
d x is detected, and the capturing signals become from x, y, a p to x <- x + d x , y\ 

15 a' p <- a p . The updated pieces of information x, y, a p are each added with a 
prime [ 4 ] to distinguish between them and the yet^to-be updated pieces of 
information. In Fig. 4, d x = D x . 

The captured video signal extracted from the omnidirectional camera 
2 is sent from the image sending means 22 to the camera-equipped portable 

20 terminal 3 via the image capturing device 1 . Accordingly, the image to be 
displayed on the image display means 32 of the camera-equipped portable 
terminal 3 also changes. That is, the desired partial object in the panoramic 
object changes in accordance with the camera work (direction and/or zoom 
operation) of the camera-equipped portable terminal 3, and the captured video 

25 signal of the changing partial object is reproduced and displayed on the image 
display means 22. 

The zoom amount a p will be described; at the time of the maximum 
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zoom amount a PM , the image extracting means 24 extracts pixel signal of the 
image sensor device 7 (Fig. 2), 21 in the x-direction and 2J in the y-direction. 
When the zoom amount is a p , 21 and 2J signals are extracted in the x- and 
y -directions, respectively at intervals of a PM /a p pixels. In this way, when the 
5 zoom amount a p is large, the captured video signal of the partial object is 

extracted from a narrow area in the image sensor device 7, whereas when a p is 
small, the captured video signal of the partial object is extracted from a wide 
area, and the image that is displayed on the image display means 32 of the 
camera-equipped portable terminal 3 is also zoomed in/out with a change in 

10 the zoom amount a p . 

The image relay means 15 of the image capturing device 1 sends the 
captured video signal received from the remote image sensing device 2 to the 
camera-equipped portable terminal 3 after a converting part 15a (see Fig. 4) 
introduces changes into the video signal according to the rating of the 

15 camera-equipped portable terminal 3 and the specs of the portable-associated 
transmission line 3, such as conversion of the screen size of the captured 
video signal to the screen size of the image display means 32 and conversion 
of amplitude (luminance) resolution or compression coding for each pixel. 

Referring to Fig. 6> the flow of processing in the system shown in Fig. 

20 4 will be described in brief. The camera-equipped portable terminal 3 sends 
a starting access to the image capturing device 1 (Fl). The image capturing 
device 1 sends the starting access or capturing signal (initial value) to the 
remote image sensing device 2 (F2). The remote image sensing device 2 
extracts the captured video signal of a partial object predetermined or 

25 corresponding to the capturing signal (which "captured video signal" will 
hereinafter be referred to also as "captured signal" in some cases) (F3), and 
the captured signal is sent to the image capturing device in real time (F4). 
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The image capturing device 1 relays the received captured signal to the 

camera-equipped portable terminal 3 (F5). 

The camera-equipped portable terminal 3 plays back the received 

captured signal to display the image of the partial object (F6). The image 
5 sensing of the surrounding object is performed by the camera-equipped 

portable terminal 3 (F7). The surrounding video signal by the image sensing 

operation is sent to the image capturing device 1 in real time (F8). 

The image capturing device 1 detects a change in the camera operation 

of the camera-equipped portable terminal 3 from a change in the image of the 
10 received surrounding video signal (F9), and if such a change is detected, 

generates capturing signal (F10) and sends the capturing signal to the remote 

image sensing device 2 (Fll) 

The remote image sensing device 2 changes, based on the received 

capturing signal, the video signal of the partial object to be extracted (F12). 
15 This is followed by the repetition of processing succeeding that F3 of 

extracting the captured signal according to the capturing signal, and upon 

sending an end command from the camera-equipped portable terminal 3 to the 

image capturing device 1 (F13), the image capturing device 1 relays the end 

command to the remote image sensing device 2 (F14), and the remote image 
20 sensing device 2 stops sending of the captured signal of the partial object 

(F15). 

Referring to Fig. 7, an example of the procedure of the image 
capturing device 1 will be described below. 

Step S 1 : Wait for reception of the starting access from the 
25 camera-equipped portable terminal 3. 

Step S2: Upon receiving the starting access, send the starting access or 
the capturing signal (initial value) in the previous signal memory 13a to the 
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remote image sensing device 2. 

Step S3: Decide whether a camera operation command is received. 

Step S4: If no operation command is received, receive the captured 
signal of a partial object from the remote image sensing device, and 
5 Step S5: Send the received captured signal to the camera-equipped 

portable terminal 3. 

Step S6: If the operation command is received in step S3, hold the 
command or set a flag, and receive a surrounding video signal from the 
camera-equipped portable terminal 3. 
10 Step S7: Check whether the variation detecting timing is reached, and 

if so, then 

Step S8: Perform variation detection processing by the variation 
detecting means 12 to obtain variation information. 

Step S9: Generate a new capturing signal by use of the obtained 
15 variation and the previous capturing signal, and update the previous capturing 
signal with the newly generated one. 

Step S10: Send the newly generated capturing signal to the remote 
image sensing device 2, and go back to step S4. 

Step Sll: After step S5, decide whether an operation stop command is 
20 received, and if not, then return to step S7. 

Step SI 2: If the top command is received in step Sll, erase the 
operation command held so far or reset the flag, and decide whether an 
operation resume command is received. 

Step SI 3: If the operation resume command is not received, decide 
25 whether an end command is received, and if not, go back to step S4. 

If the variation detection timing is not reached in step S7, then go to 
step S4, and if the operation resume command is received in step SI 2, then 
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hold the command and, for example, set a flag, followed by the return to step 
S6. 

With such procedure as described above, when the user sends the 
starting access from the camera-equipped portable terminal 3 to the image 
5 capturing device 1, the captured signal of the partial object is played back and 
displayed on the camera-equipped portable terminal 3; if the user wants to see 
an image of a partial object different from that being displayed or to zoom it 
in or out, the camera operation command is sent from the camera-equipped 
portable terminal 3 to the remote image sensing device 1 for a desired camera 

10 operation, and upon displaying an image of the desired partial object, the 

operation stop command is sent. Thus, the user is allowed to keep on seeing 
the image of the partial object, for example, while carrying the 
camera-equipped portable terminal 3 with him. The partial object desired to 
see can be switched, as required, simply by issuing the operation resume 

15 command. 

Upon receiving the end command in step SI 3, the image capturing 
device 1 may send the end command to the remote image sensing device 1 as 
required (S14) to cause the remote image sensing device 1 to end transmission 
of the captured signal of the partial object immediately upon receiving the end 

20 command. Another possible method is to end the transmission of the 

captured signal a certain time elapsed after the first sending of the captured 
signal of the partial object. The surrounding video signal may also be 
received in the interval between the reception of the starting access and the 
reception of the operation command as indicated by the broken line in Fig. 7. 

25 While in the above the variation detecting timing is checked in step S7 to 
thereby perform the variation detection processing, for example, at a preset 
time intervals, it is also possible to omit step S7 and perform the variation 



22 

detection processing at all times. As indicated by the broken lines in Fig. 7, 
the variation detection processing may be followed by determining whether 
the currently received surrounding image differs from the previously received 
surrounding image, and if no change is found, the procedure goes to step S4, 
5 whereas if a change is found, the procedure goes to step S9 (SI 5). 
EMBODIMENT 2 

In Embodiment 1 the omnidirectional camera is used as the image 
sensing means of the remote image sensing device 1 and the image capturing 
device 1 sends the capturing signal to the remote image sensing device to 

10 cause it to extract the partial-object captured signal, but in Embodiment 2 an 
omnidirectional camera is used as image sensing means of the remote image 
sensing device 2, and the remote image sensing device 2 is caused to send all 
captured video signals to the image capturing device 1, and the image 
capturing device 1 extracts the partial-object captured signal from the 

15 received captured video signals. 

Fig. 8 illustrate an example of the system configuration of this 
embodiment. The following description will be given only of the parts 
different from those in Fig. 4. In the remote image sensing device 2 the 
image extracting means 24 is omitted, and all the captured video signal of the 

20 omnidirectional camera 21 are sent from the video sending means 22 to the 
image capturing device 1 . The signal receiving means 23 may be left out, 
but instead provision may be made such that the remote image sensing device 
responds to the starting access from the image capturing device 1 to start 
image sensing by the omnidirectional camera 21 or start transmission of 

25 captured video signals to the image capturing device 1 and responds to the 
end command to end the transmission of the captured video signals to the 
image capturing device 1 . 
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In the image capturing device 1 there are provided remote image 
receiving means 16, image extracting means 17 and image sending means 18, 
and no capturing signal is sent to the remote image sensing device 2. The 
captured video signal from the remote image sensing device 2 is received by 
5 the remote image receiving means 16, and the capturing signal from the 
capturing signal generating means 13 is used to extract the partial-object 
captured signal by the image extracting means 17. For example, the 
received captured video signal is stored in a buffer memory 17a in the image 
extracting means 17; in this case, the video signal is stored with respective 

10 pixel signals arranged in the same pattern as that of the pixel arrangement in 
the image sensor device 7 shown in Fig. 2, for instance. By the same 
method as that for extracting the partial-object captured signal by the image 
extracting means 24 in the remote image sensing device 2 in Embodiment 1, 
the partial-object captured signal is read out from the buffer memory 17a 

15 based on the capturing signal in the previous memory 13a (see Fig. 5) in the 
capturing signal generating means, in this example, based on reference pixel 
position signals x, y and the zoom-in/out signal a p . The thus read-out 
partial-object captured signal is sent from the image sending means 18 to the 
camera-equipped portable terminal 3. In this case, processing of the 

20 captured signal, such as the screen size conversion and compression 
processing, may be performed in the converting part 18a before its 
transmission as in Embodiment 1 . The starting access and the end command 
may be sent, as required, by the signal sending means 14 to the remote image 
sensing device 2. This embodiment is identical with Embodiment 1 except 

25 the above. 

Fig. 9 shows an example of the procedure of the image capturing 
device 1 in Embodiment 2. As is the case with the procedure depicted in Fig. 
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7, steps SI, S2 and S3 are performed. If no command for camera operation 
is received in step S3, the captured video signal is received from the remote 
image sensing device 2 in step S4, then in step S16 a partial-object captured 
signal is extracted from the received captured video signal based on the 
5 variation information stored in the previous signal memory 13a, and in step 
S 17 the thus extracted partial-object captured signal is sent to the 
camera-equipped portable terminal 3. 

After sending the captured image, the procedure goes to step Sll , in 
which it is determined if a camera operation stop command is received as in 

10 Embodiment 1, then steps S12 and S 13 are sequentially performed, and if no 
end command is received, the procedure goes to step S4. This embodiment 
is identical with Embodiment 1 except the above and produces the same 
working-effect as that obtainable with the latter. The flow of processing as 
the system is the same as in Embodiment 1 except that: the remote image 

15 sensing device 2 sends all the captured video signal to the image capturing 
device 1 in step F3; step F12 is omitted; and the image capturing device 1 
performs the processing shown in Fig. 9. 
EMBODIMENT 3 

Embodiment 3 uses a plurality of digital video cameras as image 

20 sensing means of the remote image sensing device 2, and the image capturing 
device 1 captures a video signal of a partial object in a panoramic object by 
extracting it in the remote image sensing device 2. 

In Fig. 10 there are shown, by way of example, relationships between 
the remote image sensing device 2 and the panoramic object 6 in Embodiment 

25 3. In this example, camera devices 2 X to 2 8 having eight video cameras, as 
the remote image sensing device 2, are set over 360 degree with their shooting 
directions held at equal angular intervals. The viewing angles 0 of the 



25 

camera devices 2 X to 2 8 each chosen to be a little larger than the angular 
distance of 45 degrees in this example so that the fields of adjacent camera 
devices partly overlap to permit image sensing continuous partial objects of 
the panoramic object. The panoramic object 6 forms a cylindrical surface at 
5 the same distance from the remote sensing device 2, but in the interests of 
easy understanding of partial objects, the panoramic object 6 is shown with its 
lower diameter smaller than the upper one. 

For example, the camera device 2 X shoots the partial object 61, and the 
camera device 2 2 shoots the partial object 62 in the direction 45 degrees away 

10 to the right from the direction of shooting the partial object 61. With this 
remote image sensing device 2, it is possible to shoot any of the partial 
objects into which the panoramic object 6 is divided equally by the number 2 n 
of camera devices used. Incidentally, the number of camera devices is not 
limited specifically eight. The angular distance of the shooting direction 

15 needs not always be equal, that is, when the panoramic object 6 contains a 
part unnecessary to see, the camera devices are so directed as not to shoot 
such an unnecessary part. 

An example of the system configuration of Embodiment 3 is 
illustrated in Fig. 11. A description will be given only of the parts different 

20 from those in the Fig. 4 configuration. The remote image sensing device 2 
has a plurality of camera devices 2 X to 2 N disposed as shown in Fig. 10, for 
instance. The camera devices 2 X to 2 N have the same construction, which 
will be described below in connection with the camera device 2 X . The image 
extracting means 24 is used as in the case of Fig. 4, but its construction and 

25 processing (operation) differs from the image extracting means 24 in Fig. 4. 
That is, the capturing signal received from the image capturing device 1 is 
composed of camera identification information IDp and, if necessary, a zoom 
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parameter a P or/and a position parameter y. Accordingly, the image 
extracting means 24 is provided with a storage part 24a for storing a P , y and 
ID storage part 24b for storing the camera identification information ID1 of 
the camera device 2 X . When either one of a p and y is not used, the storage 
5 part 24a is omitted. 

When the camera identification information IDp in the capturing 
signal received by the signal receiving means 23 matches the camera 
identification information, ID1 in this example, stored in the ID storage part 
24b, the camera 21 of the camera device 2\ begins shooting and image 

10 capturing processing is performed; otherwise, the camera 21 is always held in 
shooting state and images are captured from the camera 21. The stored 
contents a P and y in the storage part 24a are updated with a P and y in the 
capturing signal, respectively. 

The image extraction processing can be performed substantially in the 

15 same manner as in the case of Embodiment 1, but when a P and y are not used, 
the image signal output from the camera 21 is captured as a partial-object 
captured signal from the camera 21 and input to the image sending means 22. 
The image sensor device 7 of the camera 21 is rectangular, for instance, as 
depicted in Fig. 12; the breadth is defined by the number of pixels, 2(a PM /a PS )I, 

20 which is a value obtained by multiplying the number of horizontal 

(x-direction) pixels of the image to be displayed, 21, by a value (a PM /a PS ) 
which is obtained by dividing the maximum value a PM of a P by the minimum 
value a s of a P , and th length is defined by the number of pixels, 
2(a PM /a PS ) Ay M J, that is a value obtained by multiplying the number of 

25 vertical (y-direction) pixels, 2J, of the image to be displayed by 2(a PM /a PS ) and 
further multiplying by the number of pixels, Ay M , for the maximum tilt angle 
(of elevation/depression). Conversely, the minimum value a PS of a P is 
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determined by the number of pixels in the breadthwise direction of the device 
7, and the maximum tilt angle (of elevation/depression) Ay M is determined by 
the minimum value a PS based on the number of pixels in the lengthwise 
direction of the device 7. 
5 For example, if the parameters stored in the storage part 24a are a P and 

y, signals of pixels ((a PM /a P )i, y+(a PM /a P )j) (i = 0, ±1, ±1, j = 0, ±1, ±J) 
captured as a partial-object captured signal from an area 71 of a breadth 
2(a PM /a P )I and a length 2(a PM /a P )J with its center at a position y on a bisector 
(center line) of the device 7 in the x-direction in Fig. 12, and when the 

10 parameters are updated with a P and y\ signals of pixels ((a PM /a P ')i, 

y ,+ (a PM /a P ')j) are captured as a partial-object captured signal from an area 72 
of a breadth 2(a PM /a P ')I and a length 2(a PM /a P ')J with its center at a position y* 
on the center line. 

As described above, in the remote image sensing device 2 the 

15 partial-object captured signal is taken out from any one of the camera devices 
2i to 2 N in accordance with the capturing signal sent from the image capturing 
device 1, and the signal is sent by the image sending means 22 to the image 
capturing device 1. The remote image sensing device 2 and the image 
capturing device 1 may be interconnected as shown in Fig. 11, in which the 

20 camera devices 2i to 2 N are connected via partial lines to the image capturing 
device 1, but the remote image sensing device 2 and the image capturing 
device 1 may also be connected via one two-way transmission line 4 as 
depicted in Fig. 13. That is, the capturing signal from the image capturing 
device 1 is received by the signal receiving means 23, then the received 

25 capturing signal is distributed by dividing means 26 to the image extracting 
means 24 of each of the camera devices 2 X to 2 N , then the partial-object 
captured signals from the respective cameras 21 of the camera devices 2 X to 
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2 N (normally, from only one of them) are combined by combining means 27, 
and the combined partial -object captured signal is send by the image sending 
means 22 to the image capturing device 1 . 

Turning back to Fig. 11, in the image capturing device 1 direction 
5 information corresponding to the shooting direction of each of the camera 
devices 2 { to 2 N is stored in camera direction storage means 19 in 
correspondence to the camera identification information IDi of each camera 
device. For example, an angle between the shooting directions of adjacent 
camera devices is stored. Fig. 14 shows an example. In Fig. 14, the 

10 number N of camera devices 2 N is 8, and respective pieces of camera 

identification information of the camera devices 2 X to 2 N are represented by 
IDI to ID8. The angular distance Axl between the camera devices 2 X and 2 2 
in the shooting direction (a value in terms of number of pixels of the captured 
image on the display surface; the same applied to the following description) is 

15 stored for ID1ID2, and the angular distance Ax2 between the camera devices 
2 2 and 2 3 in the shooting direction is stored for ID2ID3, and the angular 
distances between the other adjacent camera devices are similarly stored. 

The capturing signal generating means 14 somewhat differs in 
construction and processing (operation) from the counterpart in Fig. 4. In 

20 Fig. 14 there is shown an example of its functional configuration. In this 
example the variation detecting means 12 is shown to detect, as the variation 
information, the direction information d x , d y and the zoom information a by 
such configuration and method as referred to previously with reference to Fig. 
5. In the previous signal memory 13a, the previous camera identification 

25 information IDp, the y-direction position y and the zoom amount a P are stored. 
As their initial values, predetermined camera identification information IDI is 
set for the identification information IDp, y = 0, and the zoom amount a P is 
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set at an intermediate value between its maximum and minimum values. 

The information d x is input to a camera identifying part 13b. The 
processing in the camera identifying part 13b will be described below with 
reference to Fig. 15. If the sign of d x is positive (SI), the angle Ax +1 between 
5 the camera shooting direction in the previous camera identification 

information IDp and the camera shooting direction in the current camera 
identification information IDp+ 1 is read out of the camera direction storage 
means 19 using IDpIDp+1 as an address (S2). A check is made to 
determine whether d x is equal to or greater than Ax+1 (S3), and if it is equal to 

10 or greater than Ax +1 , IDP+1 is output (S4), and if it is not equal to or greater 
than Ax +1 , IDp is output intact (S5). 

If the sign of d x is not positive in step SI, then the angle Ax^i between 
the camera shooting directions in IDp-1 and IDp is read out of the storage 
means 19 using IDp- 1 IDp as an address (S6), then a check is made to 

15 determine whether the absolute value of d x is equal to or greater than Ax_! 
(S7), and if so, IDp-1 is output (S8), and if it is not equal to or greater than 
Ax_i, IDp is output intact (S9) 

Where the angular distance between the shooting directions of 
adjacent camera devices is small, or where the shooting direction of the 

20 camera-equipped portable terminal 3 is suddenly changed a relatively large 
angle, steps S4 and S8 are omitted as indicated by the broken lines in Fig. 15; 
if d x is equal to or greater than Ax +i in step S3, the angular distance from the 
shooting direction of the adjoining camera, that is, the angular distance Ax +1 
between the camera shooting directions in IDp+1 and IDp+2, is read out from 

25 the storage means 19 (S10), then a check is made to determine whether d x is 
equal to or greater than Ax +1 + Ax +2 (Sll), and if so, IDp+2 is output (SI 2), 
and if it is not equal to or greater than Ax +i + Ax +2 , IDp+1 is output (SI 3). 
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When |d x | is equal to or greater than Ax_j in step S7, the angular 
distance from the shooting direction of the adjoining camera, that is, the 
angular distance Ax_ 2 is read out of the storage means 19 using IDp-2-IDp-l 
(S14), and a check is made determine whether |d x | is equal to or greater than 
5 Ax_i + Ax_ 2 (S15), and if so, IDp-2 is output(S16), and if not so, IDp-1 is 
output (S17). Similarly, the angular distances from camera shooting 
directions of further distant cameras can be decided. 

Where the angular distances between the shooting directions of 
adjacent cameras are all equal as in the Fig. 10 example, only the number of 

10 pixels Ax corresponding to that angle, 45 degrees in this example, is stored in 
the camera direction storage means 19 as indicated by the broken lines, and 
steps S2 and S6 are omitted; in step S3 a check is made to determined if 
Ax<d x , and in step S7 a check is made to determine if Ax<|d x |. For 
large-angle camera operation, steps S4, S8, S10 and S14 in Fig. 15 are 

15 omitted as indicated by the broken lines, then in step Sll a check is made to 
see if 2Ax<d x , and in step S15 a check is made to see if 2Ax<|d x |. In this 
instance, too, the angular distances from camera shooting directions of further 
distant cameras can similarly be decided. 

Turning back to Fig. 14, the capturing signal generating means 13 will 

20 be described below. As for the input pieces of information d y and a, d y is 
added with the previous information y in the adding part 13c and a is 
multiplied by the previous information a P in the multiplying part 13d as 
described previously with respect to Fig. 13. The outputs from the 
respective parts 13b, 13c and 13d are provided to the signal sending means 14, 

25 while at the same time the contents of the previous signal memory 13a are 
updated in the same manner as described previously. For the respective 
inputs, the discarding parts 13e, 13f and 13g may be provided. In this 
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instance, when d x is equal to or smaller than a preset value in the discarding 
part 13e, the present value IDp (IDp in the memory 13a) is output without the 
processing in the camera identifying part 13b. 

In the camera direction storage means 19 there may be prestored the 
5 angles of shooting directions of the camera devices with respect to the 

direction shooting of a predetermined one of them as depicted in Fig. 16, for 
instance. In the example of Fig. 16, the shooting direction of the camera 
device of the camera identification information ID4 is used as a standard of 
reference and the angles of the shooting direction of each of the other seven 

10 camera devices with respect to this reference direction is stored as the number 
of pixels in one frame of the captured image. In this case, in the processing 
shown in Fig. 15, the angles in IDp and IDp+1 are read out, for example, in 
step S2 and it is decided whether d x is equal to or greater than the difference 
between the two angles. 

15 According to the usage pattern, when the direction of the partial object 

that the user wants to see first is predetermined, it is necessary that the 
respective pieces of camera identification information be made to correspond 
to north, south, east and west directions, for example, north, north-east, east, 
. . . directions. The correspondences between the camera identification 

20 information and the north, south, east and west directions may be defined by 
predetermining the shooting direction of the camera device of each camera 
identification information and placing the remote image sensing device 2 
accordingly. In some case, however, such placement of the remote image 
sensing device 1 is time-consuming. To avoid this, as shown in Fig. 11, 

25 camera information measuring means 25 is provided in each of the camera 
devices 2i to 2 N , and the angle of the shooting direction with respect to true 
north is measured by a magnetic compass or similar direction sensor 25a of 
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the camera information measuring means 25 to obtain the information about 
the shooting direction of each camera device in north, south, east, or west 
direction. In the illustrated example, a tilt angle sensor 25b as by a 
gravitational accelerometer is also provided, by which is detected a value Ay 
5 that represents, in terms of the number of pixels on the frame of the captured 
image, the angle of the shooting direction to the horizontal plane, that is, the 
angle (angle of elevation/depression) of the y-axis of the image sensor device 
7 of the camera 21. The north, south, east or west direction and the tilt angle 
Ayn measured by the camera information measuring means 25 are sent, 

10 together with the camera identification information IDn of the camera device 
2 n (n = 1, . . ., N), to the image capturing device 1 by the signal 
sending/receiving means 23 as indicated by the term in parentheses in Figs. 11 
and 13. The image capturing device 1 receives from each camera device 2 n 
its identification information IDn, north, south, east or west direction and tilt 

15 angle Ayn by the signal sending/receiving means 15, and stores them in the 
camera direction storage means 19 in correspondence to the identification IDn 
as shown in Fig. 16, for instance. 

When the user makes a starting access to the image capturing device 1, 
the image capturing device 1 reads out the identification information of the 

20 camera, ID3 in the Fig. 16 example, whose shooting direction is a 

predetermined one of the north, south, east and west directions, for example, 
north direction — ID3 in the Fig. 16 example — and the elevation/depression 
angle Ay3 , and sends these read-out values to the remote image sensing 
device 2, if necessary, together with the initial value a P of the zoom in/out 

25 parameter (zoom value). The initial value a P may be prestored in the storage 
part 24a of the image extracting means 24 in each of the camera devices 2 X to 
2 N . The elevation/depression angle Ayn is subtracted from the output from 
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the adding part 13c in a correcting part h in Fig. 12, and is fed to the signal 
sending means 14. The updating of y in the previous signal memory 13 is 
performed using the value yet to be corrected in the correcting part 13h. For 
example, where the user wants to see a partial-object image with reference to 
5 the horizontal direction, if the camera shooting direction is a little upward 
with respect to a horizontal plane, the partial-object image formed on the 
image sensor device 7 of the camera 21 is shifted upwards from a 
partial-object image in the horizontal direction by a value corresponding to 
the upward shooting direction, that is, by Ayn. Hence the subtraction of Ayn 

10 in the correcting part 13h provides a partial -object image with reference to the 
horizontal direction. Incidentally, the initial value of y is set at y = 0. 

As described above, in response to the capturing signal from the image 
capturing device 1 a partial-object video signal is sent from the camera device 
2 n to the image capturing device 1 to capture the signal, which is relayed and 

15 sent to the camera-equipped portable terminal 3 as in the case of Embodiment 
1; accordingly, it can easily be understood that the user can see the panoramic 
object directly and select its partial object while operating the camera of the 
portable terminal 3 as in Embodiment 1 . The procedure of the image 
capturing device 1 in this instance is the same as shown in Fig. 7. The 

20 procedure in this case may also be changed such that upon receiving the 
starting access, the image capturing device sends it to the remote image 
sensing device 2, which responds to the starting access to send a partial-object 
captured signal from a predetermined camera device 2 n to the image capturing 
device L 

25 EMBODIMENT 4 

In the case of using the plurality of camera devices 2 X to 2 N as the 
remote image sensing device 2, too, the image capturing device 1 can extract 
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a partial-object captured signal from the video signal received from the 
remote image sensing device 2 as in the case of Embodiment 2. 
Embodiment 4 concerns this operation, and its differences from Embodiment 
3 will be described with reference to Fig. 17. 
5 In the remote image sensing device 2, the image extracting means 24 

of each of the camera devices 2i to 2 N is omitted, and the video signal from 
each camera 21 is sent by the image sending means 22 to the image capturing 
device 1 . In this instance, the camera identification information IDn stored 
in the ID storage part 24b, for instance, is added to the video signal of the 

10 camera 21 to enable the image capturing device 1 to recognize the camera 
device 2 n from which the video signal is received. Incidentally, if input 
terminals of the remote image receiving means 16 in the image capturing 
device 1 are predetermined corresponding to the camera devices 2i to 2 N , 
there is no need for sending the camera identification information IDn. The 

15 image sending means 22 may be provided for each of the camera devices 2i to 
2 N , but it may also be provided in common to them. Furthermore, the 
camera information measuring means 25 may be provided as required. 

In the image capturing device 1 the image relay means 15 is omitted, 
and in the image extracting means 17 a selecting part 17b selects that one of 

20 the video signal from the camera device 2i to 2 N which corresponds to the 
camera identification information IDp contained in the capturing signal from 
the capturing signal generating means 13. In other words, that one of the 
received signal whose camera identification information IDn matches IDp is 
selected. This selection may be made by the image extracting means 17 

25 itself. Further, the image extracting means 17 has the buffer memory 17a 
that is the same as in Fig. 8, and in this buffer memory 17a the video signal is 
stored with its pixel signals arranged in the same pattern as that of the pixel 
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arrangement in the image sensor device 7 shown in Fig. 12, for instance; and 
the image extracting means 24 extracts the partial -object captured signal in 
accordance with a P and/or y in the capturing signal as described previously 
with reference to Fig. 12. 
5 When the capturing signal contains only IDp, the buffer memory 17a 

may be dispensed with. The selection of the video signal may be done by a 
selecting part 16a in the image receiving means 16 in place of the selecting 
part 17a; alternatively, as described previously, the receiving terminals of the 
image receiving means are predetermined corresponding to the camera 

10 devices 2\ to 2 N , respectively so that the receiving terminal corresponding to 
the information IDp is selected to select the received image signal. 

On being accessed for starting from the camera-equipped portable 
terminal 3, the image capturing device 1 captures a partial-object video signal 
corresponding to predetermined camera identification information or 

15 predetermined one of the north, south, east and west directions, and sends the 
captured signal to the camera-equipped portable terminal 3. The procedure 
of the image capturing device 1 in the system of Embodiment 4 is the same as 
the procedure shown in Fig. 9. Hence, it will easily be understood that 
Embodiment 4 also produces the same working-effect obtainable with 

20 Embodiments 1 to 3. 
EMBODIMENT 5 

Embodiment 5 uses, as the image sensing means of the remote image 
sensing device 2, a video camera mounted on a panhead. Fig. 18 illustrates 
an example of the system configuration of this embodiment. A description 

25 will be given of differences from Fig. 4 configuration. 

The image sensing means 21 of the remote image sensing device 2 is a 
video camera 28 mounted on a panhead 27 as shown in Fig. 19; the shooting 
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diction of the camera 28 can be changed by a control signal from a remote 
location, and the field angle, i.e., what is called zoom amount, can also be 
changed by the lens. The image sensing means 21 will hereinafter be 
referred to as a panhead camera 21. To the image extracting means 24 are 
5 input, as capturing signals, an azimuth signal 0, an elevation/depression angle 
signal (J) and a zoom signal Z from the image capturing device 1. An 
azimuth control signal 24c controls an azimuth angle control mechanism 24d 
according to the azimuth signal 0, causing the azimuth angle of the shooting 
direction of the camera 21 to become 0. An elevation/depression control 

10 part 24e controls an elevation/depression control mechanism 24f according to 
the elevation/depression angle signal <|), causing the elevation/depression 
angle of the shooting direction of the camera to become (|). A zoom control 
part 24g controls a zoom control mechanism 24h according to the zoom signal 
Z, causing the zoom amount of the camera 21 to become Z. These control 

15 parts 24c, 24e, 24g and control mechanisms 24d, 24f, 14h may be the same as 
those built in the panhead 27 of a commercially available remotely controlled 
panhead camera. 

Thus the partial -object image signal in the panoramic object 6 
corresponding to the capturing signals is taken out from the camera 21 and 

20 sent to the image capturing device 1 . The remote image sensing device 2 
can also be provided with the camera information measuring means 25 
described previously with respect to Fig. 11. 

In the image capturing device 1, the capturing signal generating means 
13, such as shown in Fig. 5, generates signals z, x and a P based on the 

25 variation information detected by the variation detecting mean 12, and these 
signals x, y and a P are converted by a converting part 13j to direction signals 
of the azimuth signal 0 and the elevation/depression angle signal ((), and a 
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zoom change signal of the zoom signal Z, which are sent to the remote image 
sensing device 2. 

Upon sending a starting access from the image capturing device 1 to 
the remote image sensing device 2, the remote image sensing device 2 sends 
5 to the image capturing device 1 a partial-object video signal obtained by 
shooting in a preset reference direction and with a reference zoom amount. 
In the remote image sensing device 2 camera information measuring means 
25 is provideod, and in the case where the image capturing device 1 responds 
to a starting access to obtain a partial-video signal in a predetermined 

10 direction, for example, in the north direction and in the horizontal direction, 
an initial signal generating part 13 k generates initial capturing signals. That 
is, the azimuth angle 0j and elevation/depression angle fa measured by the 
camera information measuring means 25 of the remote image sensing device 
2 are stored in storage parts 13 m and 13 n , respectively, and 

15 direction-containing angle signals of differences between the stored angles 0i, 
<j>i and predetermined reference azimuth angle and reference 
elevation/depression angle are calculated in reference calculating parts 13p 
and 13q, and these signals are sent as initial capturing signals to the remote 
image sensing device 2. That is, the difference between the current shooting 

20 direction of the panhead camera 21 and the reference direction is sent as the 
capturing signal to the remote image sensing device 2, which responds to the 
capturing signal to control the panhead camera 21 to change its shooting 
direction by the above^nentioned difference. As regards the zoom amount Z, 
the initial capturing signal is zero in difference. In the case of effecting such 

25 control, the capturing signals based on the variation information detected by 
the variation detecting means 12, which indicate the current conditions of the 
panhead camera 21, that is, only variations in the shooting direction and in the 
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zoom amount, are also sent to the remote image sensing device 2. 
Accordingly, as depicted in Fig. 18, for instance, the pieces of variation 
information d x , d y and a detected by the variation detecting means 12 are 
directly input to a converting part 13j of the capturing signal generating 
5 means 13, and respective converted outputs are sent as generated capturing 
signals to the remote image sensing device 2. The remote image sensing 
device 2 controls the azimuth control mechanism 24d, the 
elevation/depression control mechanism 24f and the zoom amount control 
mechanism 24h by the variations represented by the received capturing 

10 signals corresponding to them, respectively. 

As described above, not only in the case of using a predetermined one 
of the north, south, east and west directions as a reference but also in the case 
of using the initial condition of the panhead camera as a reference, it is also 
possible to: generate capturing signals representing variations corresponding 

15 to the variation information detected by the variation detecting means 12; 
send the capturing signals to the remote image sensing device 2; and effects 
control in the remote image sensing device 1 to change the current direction 
or/and zoom amount of the panhead camera 21 by the amounts of variation 
indicated by the received capturing signals. The he method in which the 

20 capturing signals representing only variations based on the variation 

information is generated and sent to the remote image sensing device 2 is also 
applicable to Embodiments 2 and 4. In this instance, in Embodiment 1, for 
example, the same capturing signal generating means as that 13 in Fig. 5 is 
provided in the remote image sensing device 2. 

25 EMBODIMENT 6 

Embodiment 6 is so adapted as to permit a plurality of users to use the 
remote image sensing device 2. For example, as depicted in Fig. 20, the 
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image capturing device 1 is connected to a communication network 5 0 , and 
plural camera-equipped portable terminals 3 U 3 2 , . . 3 M are connected 
directly to the communication network 5 0 or connected to the communication 
network 5 0 via another network, constituting portable-associated transmission 
5 lines 5i, 5 2 , . . . , 5 M connected to the image capturing device 1 . The image 
capturing device 1 is connected to the remote image sensing device 2 via the 
remote transmission line 4. The remote transmission line 5 may be a 
dedicated line or a line passing through the communication network 5 0 . 

A description will be given first of the case where the remote image 

10 sensing device 2 is similar in configuration to that described previously with 
respect to Embodiment 2. An example of the functional configuration of the 
image capturing device 1 in this example is shown in Fig. 21. There is 
provided: a plurality S of lines for connection with a plurality of 
camera-equipped portable terminals 3, that is, a plurality of surrounding 

15 image receiving means 11 1 to ll s ; receiving buffer memories 12a! to 12a s ; a 
previous frame memory 12b s having S areas for storing surrounding image 
signals each corresponding to one of line numbers s (s = 1, 2, . . ., S); a flag 
memory 92 for storing a flag indicating whether the corresponding camera is 
being operated, in correspondence to each line number s; an update 

20 instructing part 12d s corresponding to each line number s; one or more 
variation detecting parts 12c (only one of which is shown); one or more 
capturing signal generating means 13 (only one of which is shown); image 
sensing means 18 s pairing off with the surrounding image receiving means ll s 
of each line; remote image means 16; one or more image capturing means 17 

25 (only one of which is shown); and control means 98. In the previous signal 
memory 13a there are provided areas where to store the previous capturing 
signal (currently used) for each line number s. The control means 93 is 
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provided with a memory having stored therein a program necessary for 
controlling a microprocessor or CPU (Central Processing Unit) to function as 
the image capturing device 1, and executes the program to effect a read and a 
write of each memory and operate the respective parts in a sequential order. 
5 The functions of some of the functional blocks in Fig. 21 are performed the 
control means 93 itself. 

A description will be given below of the case where a certain 
camera-equipped portable terminal 3 uses a line s until completion of 
reception of a partial-object image signal after connecting to the surrounding 

10 image receiving means ll s of the line number s to make a starting. Upon the 
starting access being made to the surrounding image receiving means ll s , the 
capturing signals are read out of that area in the previous signal memory 13a 
corresponding to the line s, and the read-out capturing signals are used to 
extract, by the image extracting means 17, a partial -object video signal from a 

15 panorama video signal received by the remote image receiving means 17, and 
the extracted partial-object video signal is sent to the camera-equipped 
portable terminal 3 by the image sending means s of the line number s. In its 
initial state the previous signal memory 13a has stored in its all areas 
capturing signal, x 0 , yo and a P0 in this example, for predetermined partial 

20 objects. In the camera-equipped portable terminal 3, upon receiving the 
video signal from the image capturing device 1, the received video signal is 
played back and displayed by the image display means 32. The operation 
for capturing the partial -object video signal in the image capturing means 17 
is carried out in the same manner as described previously in respect of 

25 Embodiment 2. 

Upon receiving a camera operation command by the surrounding 
image receiving means ll s of the line s, a flag is set for the line number s in 
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the flag memory 92, that is, the flag is switched from "0" to "1" (the flag 
being initially set at "0"), then the timer of the update instructing part 12d s is 
set, and the surrounding video signal received by the surrounding image 
receiving means ll s is stored in the receiving buffer memory 12a s . 
5 Upon each issuance of an update instruction from the update 

instructing part 12d s , the variation information d x , d y , a is detected by the 
variation detecting part 12c from the both received surrounding video signals 
in the receiving buffer memory 12a s and the previous frame memory 12b, 
then the capturing signal generating means 13 generates a capturing signal by 

10 processing the previous generating signal in the area s of the previous signal 
memory 13a based on the detected variation information, then the capturing 
signal in the area s of the previous signal memory 13a is updated with the 
newly generated capturing signal, and a partial-object video signal by the 
image capturing means 17 is sent by the image sending means 18 s to the 

15 camera-equipped portable terminal 3. The video signal in the receiving 

buffer memory 12a s is transferred to the area s of the previous frame memory 
12b. 

Upon receiving a camera operation stop command by the surrounding 
image receiving means ll s , the flag of the line number s in the flag memory 

20 92 is reset to "0," stopping the operation of the update instructing part 12d s . 
Based on the capturing signal in the area s of the previous memory 13a, the 
partial-object video signal is captured, and it is send by the image sending 
means 18 s on a real time basis. Upon receiving an operation resume 
command by the surrounding image receiving means ll s , the flag of the line s 

25 in the flag memory 92 is set to "1" to put the update instructing part 12d s in 
operation, followed by the same processing as in the case of having received 
the camera operation command. When an end command is received by the 
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surrounding image receiving means ll s , the contents in the previous buffer 
memory 12a s the area s of the previous frame memory 12b and the area s of 
the previous signal memory 13a are erased, and the update instructing part 
12d s is stopped from operation as required, and the flag of the line s in the flag 
5 memory 92 is reset to "0." 

For fast processing, provision may also be made to establish 
connection between the image capturing device 1 and the camera-equipped 
portable terminal 3 for each signal transmission and reception therebetween 
until completion of the image capturing operation after sending of the starting 

10 access from the camera-equipped portable terminal 3 m (M = 1, 2, . . M) to 
the image capturing device 1 . In such an instance, the camera-equipped 
portable terminal 3 m attaches terminal identification information Idm to each 
signal to be sent. The terminal identification information idm used is, for 
example, the telephone number or address of the camera-equipped portable 

15 terminal 3 m or, if necessary, the equipment number of the camera-equipped 
portable number 3 m , the user's name or so. A processing number memory 
9 1 is provided to define the correspondence between the terminal 
identification information Idm and the processing number s (s = 1, 2, . . ., S) of 
the circuit number. On receiving the starting access, the terminal 

20 identification information Idm and an idle processing number s (the number 
indicating a line currently not used) are stored in the processing number 
memory 91 in correspondence to each other. Thereafter, upon each 
reception of a signal from the camera-equipped portable terminal 3 m , the 
image capturing device 1 uses its terminal identification information Idm to 

25 read out the processing number s from the processing number memory 91, 
and performs processing corresponding to the read-out processing number s. 
That is, based on the terminal identification information Idm, the flag of the 
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processing number s in the flag memory 91 is controlled, and the operation of 
the update instructing part 12d s is controlled; based on the surrounding video 
signal, the areas s in the receiving buffer memory 12a s and the previous frame 
memory 12b are utilized, and the capturing signal in the previous signal 
5 memory 13a of the capturing signal generating means 13 and the signal in the 
memory of the image capturing means 17; and sending of the partial-object 
video signal is performed by reading out the terminal identification 
information Idm corresponding to the processing number s in the processing 
number memory 91 and connecting an idle image sending means 18 s to the 

10 camera-equipped portable terminal 3 m of the above-mentioned read-out 

terminal identification information Idm. As described above, according to 
Embodiment 6, a plurality of users can see their desired partial -object images 
via one remote image sensing device 1 by simultaneously operating their 
camera-equipped portable terminals independently of one another. 

15 Incidentally, when processing is congested, pluralities of variation detecting 
parts 12c, capturing signal generating means 13 and image capturing means 
17 are used at the same time. 

Where the remote image sensing device 1 used is the same as that 
shown in Fig. 10, too, a plurality of users can use one remote image sensing 

20 device 1 at the same time. That is, in the configuration depicted in Fig. 21 
the remote image receiving means 16, the capturing signal generating means 
13 and the image capturing means 17 are configured as shown in Figs. 14 and 
17; further, to perform simultaneous processing for the plurality of 
camera-equipped portable terminals 3 m , the image capturing device 1 has, as 

25 shown in Fig. 21, the plurality of surrounding image receiving means ll s , the 
plurality of frame memories 12a s , the plurality of update instructing parts 12d s , 
the previous frame memory 12b having a plurality of storage areas, the 
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previous signal memory 13a having a plurality of storage areas, the plurality 
of image sending means 18 s , the flag memory 92, and the control means 93. 
And, as noted in the parentheses in the previous signal memory 13a, the 
camera identification IDp is used in place of the signal x in the capturing 
5 signal, and processing for each camera-equipped portable terminal 3 m is 
performed in the same manner as in the case of using the omnidirectional 
camera, but the partial-object video signal capturing operation is carried out 
as described previously with reference to Embodiment 4. In this case, too, it 
can easily be understood that the plurality of users can see simultaneously on 

10 the plurality of camera-equipped portable terminals 3 m their desired 
partial-object images of the video signals in plural directions that are 
simultaneously received from the plurality of camera devices 2\ to 2 N . 

The systems of Embodiments 1 and 3 can also be adapted to 
simultaneously capture partial -objects images by the plurality of 

15 camera-equipped portable terminals 3 m . In this instance, the image 
capturing means 17 in Fig. 21 is omitted, and the capturing signal for the 
camera-equipped portable terminal 3 m , generated in the image capturing 
device 1, is attached with, for example, the processing number s for 
identification use, and sent to the remote image sensing device 2; in the 

20 remote image sensing device 1, in the case of the Fig. 4 example, a storage 
parts 24a for storing capturing signals for respective processing numbers are 
each provided in the image extracting means 24 as indicated by the broken 
line in Fig. 20, then the capturing signals in the storage parts 24a 
corresponding to the received processing numbers are updated, then 

25 partial-object video signals are extracted based on the capturing signals stored 
in the respective storage parts 24a as described previously with reference to 
Embodiment 1, and the extracted partial -object video signals, attached with 
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the processing numbers, are sent to the image capturing device 1. In the 
image capturing device 1 the partial-object video signals simultaneously 
received by the camera-equipped portable terminals 3 m , corresponding to the 
received processing numbers, are relayed and sent; the image capturing means 
5 17 is omitted. 

In the case of the system shown in Fig. 1 1, as is the case with the 
above, the storage part 24a for storing the capturing signal for each processing 
number is provided in the image extracting means 24 of the remote image 
sensing device 1 ; the capturing signal in the storage part 24a,corresponding to 

10 the received processing number, is updated with the capturing signal received 
simultaneously with the processing number, then based on each capturing 
signals IDp, x, a P stored in the storage part 24a, the partial-object video signal 
is captured from the corresponding camera device 2 n , and the captured vide 
signal is sent to the image capturing device 1, together with the processing 

15 number. 

MODIFICATIONS 

In any of the above-described embodiments, the image display means 
32 may be a fixed display for use in a personal computer or the like, as 

20 indicated by the broken lines in Figs. 4, 8, 18 and 20. In any case, however, 
the display is adapted to play back and display the partial-object video signal 
sent from the image capturing device 1 to the camera-equipped portable 
terminal. In this instance, the user operates the camera-equipped portable 
terminal 3 for camera work while seeing the image displayed on the fixed 

25 image display means 32. The camera-equipped portable terminal 3 with the 
image display means 32 fixed thereto as in the above example is very 
convenient to use since it is possible for the user to use it anywhere even 
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while walking, for example, to see his child in kindergarten. 

The panoramic object 6 needs not always to extend around over 360 
degrees, but the object 6 needs only to permit changing the shooting direction 
or adjustment of the field angle in the case of shooting the object while 
5 directly watching it. Further, as referred to previously in respect of 

Embodiment 3, the panoramic object 6 need not necessarily be continuous. 
These will readily be understood from the fact that it is sometimes desirable, 
for example, to shoot only specified parts of a 180-degree scene opening out 
before the camera or it is used as a security camera for monitoring only 

10 several doorways of a building. 

In either case, it is possible to turn the shooting direction of the 
camera-equipped portable terminal 3 from side to side to select the desired 
partial object, in the above-described example, by using only x or IDp as the 
capturing signal; further, it is possible to turn the shooting direction of the 

15 camera-equipped portable terminal 3 up and down or/and move the camera 
back and forth to select the desired partial object, in the above-described 
example, by adding y or/and a P to the capturing signal. In the above, for 
each camera operation, the variation information including its amount of 
operation is detected and the capturing signal to be generated is corrected 

20 corresponding to the variation information, but it is also possible to detect a 
variation in direction alone without detecting the degree (amount) of the 
variation and add the predetermined value d x to or subtract it from the 
previous capturing signal x. 

The occasion will sometimes arise where during shooting the 

25 surroundings by the camera-equipped portable terminal 3 a flash of powerful 
light suddenly impinges on the camera 33 by some cause, disturbing the 
surrounding vide signal. If the capturing signal is generated based on 
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variation information detected from such a disturbed surrounding video signal, 
then an utterly unexpected partial-object image is received and displayed. 
To avoid this, it is recommended to provide an erroneous detection preventing 
part in the variation detecting means 12 of the image capturing device 1 in 
5 each embodiment. An example of the erroneous detection preventing part 
will be described below in connection with Embodiment 1 . As indicted by 
the broken line in Fig. 4, hysteresis information on the variation information 
sent to the capturing signal generating means 13 is stored in a hysteresis 
storage part 12e. The hysteresis storage part is configured to always store 

10 variation information obtained by, for example, three immediately previous 
detections. For example, when variation information is detected in the 
detecting part 12c in Fig. 5, a decision part 12f decides, by comparison, 
whether the variation information greatly deviates from the continuity of the 
variation information stored in the hysteresis storage part 12e. If it is 

15 decided that the current variation information does not greatly deviates from 
the continuity, that is, if the current information is not abnormal, then the 
variation information is provided to the capturing signal generating means 13, 
and at the same time, it is stored as the latest information in the hysteresis 
storage part 12e, removing therefrom the oldest information. If the decision 

20 part 12f decides that the currently detected variation information is abnormal, 
then the variation detecting means discards the information in the rejecting 
part 12g, or waits for the next update instruction, or receives the surrounding 
video signal anew and performs variation detection processing. 

When a plurality of users simultaneously use one remote image - 

25 sensing device 1, provision may be made such that: while watching an image 
on the display surface 3a of the image display means of a user terminal 3' 
such as a personal computer, each user operates an operating part 3b by 
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means of key, lever, mouse, or track ball to capture from the panorama video 
signal a moved or/and zoomed version of a desired partial object in the 
panoramic object 6 being image sensed by the remote image sensing device 1. 
The operating part 3b uses, as d x in the afore-mentioned variation information 
5 and the aforementioned d y , an x-axis control signal and a y-axis control 

signal produced by an operation for moving a display on the display screen of 
a personal computer, for instance, and outputs a control signal for zooming 
in/out the display on the screen as a in the aforementioned variation 
information, and sends this variation information d x to the image capturing 

10 device 1. In this instance, the image capturing device 1 uses mere signal 
receiving means lis as a substitute for the surrounding image receiving 
means lis, and the capturing signal generating mean 13 generates the 
capturing signal using the variation information received by the receiving 
means lis from each user terminal 3' and the previous capturing signal x (or 

15 IDp) , y, a P in the previous signal memory 13a according to the processing 
number based on the line number of the signal receiving means lis or 
simultaneously received user identification information. The subsequent 
processing is the same as in Embodiment 6. But the capturing signal is sent 
to the user terminal 3'. 

20 Where a plural users uses the remote image sensing device 2 in 

common, the capturing of the partial -object video signal for each user can be 
carried out by the image capturing device 1 as in Embodiment 1 or 3. In 
such an instance, the image capturing device 1 generates the capturing signal 
according to the variation information d x , d y , a received for each user terminal 

25 3' in the same way as described previously, but the capturing signals are IDp, 
y , a, but as referred to near the end of the description of Embodiment 6, the 
capturing signal, for example, attached with the processing number, is sent to 
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he remote image sensing device 2, and the remote image sensing device 2 
similarly sends the partial-object vide signal corresponding to the capturing 
signal for each processing number. 

While the user terminal V has been described as being fixed like a 
5 personal computer, it is apparent that it may be a portable telephone, PDA, or 
similar portable terminal as well. That is, the user terminal is to send to the 
image capturing device 1 variation information, for example, d x , d y , a, that 
represent direction and/or zoom with respect to predetermined references so 
as to specify that partial object in the panoramic object being shot by the 

10 remote image sensing device 2. 

The image capturing device 1 used in each of Embodiments 1 to 6 can 
be implemented by a computer. That is, the program for executing the 
functions of each image capturing device 1 is installed on the computer from 
a recording medium such as a CD-ROM, magnetic disk, and a semiconductor 

15 memory, or downloaded to the computer via a communication line for 
execution by the computer. 



